feat: CheckMetadataHash extension
#1865
Conversation
* move routers to its own module * remove outdated markers * for all runtimes * remove previous tests * minor fixes
…t-v1.7.2-metadata-hash
| } | ||
| } | ||
| } | ||
| pub type UpgradeDevelopment1102 = (); |
| @@ -247,7 +247,7 @@ pub mod utils { | |||
| frame_system::CheckNonce::<T>::from(nonce), | |||
| frame_system::CheckWeight::<T>::new(), | |||
| pallet_transaction_payment::ChargeTransactionPayment::<T>::from(0), | |||
| frame_metadata_hash_extension::CheckMetadataHash::<T>::new(true), | |||
| frame_metadata_hash_extension::CheckMetadataHash::<T>::new(false), | |||
There was a problem hiding this comment.
@lemunozm Unfortunately, multiple of our integration tests fail with CheckMetadataHash enabled: https://github.com/centrifuge/centrifuge-chain/actions/runs/9665090255/job/26661355826
I am not fully sure why. Based on the build.rs, it is only enabled for --features=std,metadata-hash:
centrifuge-chain/runtime/altair/build.rs
Lines 1 to 6 in 2e8e44c
Moreover, it is enabled for on-chain-release-build:
centrifuge-chain/runtime/development/Cargo.toml
Lines 450 to 457 in 2e8e44c
I tried both running integration tests with -F metadata-hash as well as enabling it for the runtimes in the integration-tests/Cargo.toml but ran into polkadot-sdk compiler issues. So it appeared to be the incorrect approach. Upon further investigating the SDK repo, I saw all their integration tests disable the metadata hash. However, this might be due to performance reasons. Apparently, the WASM needs to be built twice if the feature is enabled (see guide).
There was a problem hiding this comment.
tried both running integration tests with -F metadata-hash as well as enabling it for the runtimes in the integration-tests/Cargo.toml but ran into polkadot-sdk compiler issues
I think this is the correct one. Nevertheless, when adding the feature, could be another missing crate from Polkadot that also requires that feature, then you get those polkadot errors. I'm going to try to overpass this
There was a problem hiding this comment.
Yeah I also think it is some missing feature for a polkadot crate. Tried out a lot and still haven't surpassed the error
error[E0433]: failed to resolve: could not find `metadata_ir` in `__private`
--> /Users/william/.cargo/git/checkouts/polkadot-sdk-cff69157b985ed76/c00ca35/substrate/primitives/api/src/lib.rs:797:1
|
797 | / decl_runtime_apis! {
798 | | /// The `Core` runtime api that every Substrate runtime needs to implement.
799 | | #[core_trait]
800 | | #[api_version(4)]
... |
827 | | }
828 | | }
| |_^ could not find `metadata_ir` in `__private`
It can be quickly checked by trying to build any runtime with on-chain-release feature. In the Polkadot SDK this works for the template runtime. For us, it doesn't for any runtime.
There was a problem hiding this comment.
I also don't understand why only ~20% of the integration tests fail. I couldn't detect a pattern yet as the errors occur for all runtimes as well as fudge and non-fudge envs.
There was a problem hiding this comment.
Fails only the test who makes a call to submit_now() or submit_later(), because those calls sign the transaction. Other calls just mutate the state
There was a problem hiding this comment.
I'm also stuck in the above error. Quite weird. It seems like frame-metadata feature is missing for sp-api. But even if you try adding it, the error still happens 🤔. It seems like who add sp-api first in the cargo tree should add the frame-metadata to it.
There was a problem hiding this comment.
Yeah I have been there as well. Good to know I havent missed anything obvious and sad this doesnt seem to work right now for us. Will probably open an issue in the SDK then.
There was a problem hiding this comment.
Yeah, Although I think this is more a "rust" problem. It seems unmaintenable that each time you add a feature to a crate, you "need" to propagate that feature to the crates that use the featured gated crate. Ideally, you need to add it even to those crates that don't require that feature. Just for the case that a possible user want to enable it.
Maybe the solution here from polkadot would be to just remove the frame-metadata feature. Include metadata_ir always. The only reason to add the feature is to avoid the double wasm compilation. But here, it could be avoided.
runtime/integration-tests/src/env.rs
Outdated
| @@ -255,7 +255,7 @@ pub mod utils { | |||
| frame_system::CheckNonce::<T>::from(nonce), | |||
| frame_system::CheckWeight::<T>::new(), | |||
| pallet_transaction_payment::ChargeTransactionPayment::<T>::from(0), | |||
| frame_metadata_hash_extension::CheckMetadataHash::<T>::new(false), | |||
| frame_metadata_hash_extension::CheckMetadataHash::<T>::new(true), | |||
There was a problem hiding this comment.
If we have this false, then IT tests work?
There was a problem hiding this comment.
Yes because it disables the metadata hash check.
There was a problem hiding this comment.
I think we can skip checking the metadata for IT. What knowledge gives us IT if that value is true?
There was a problem hiding this comment.
I wanted to make the same proposal. In the end, we expect the SDK to ensure the validity of this API.
There was a problem hiding this comment.
For signed-related things, we also mock the nonce and similar things for IT, so I think we're fine not checking that. It feels like out of scope (?)
|
During engineering sync, we decided to add this feature into the release due to the signature success on/with Polkadot Js, Polkadot Js Extension, Polkadot Vault, Talisman and as of last night, Centrifuge Apps. |
* Switch to full wasm execution * feat: use correct encoded size. * fix: lint * refactor: reduce dev session duration from 6h to 2min * chore: update centrifuge weights * chore: update dev weights * chore: update altair weights * chore: update frame_system weights * fmt * fix: re-enable frame_system benches * chore: bump spec version * fmt: revert using latest nightly * v0.11.3-rc1: revert checkmetadata ext * feat: `CheckMetadataHash` extension (#1865) * set dependency versions * upgrade libs/* * upgrade pallet-anchors * upgrade pallet-fees * upgrade pallet-bridge * migrate simple pallets * migrate pallet-order-book * migrated collator-allowlist & swaps * upgrade rewards * upgraded pallet-mux * upgrade transfer-allowlist * fix hold reason in restricted tokens * simplify set_balance removing the holding part * upgrade restricted-xtokens * upgrade some pallets * upgrade pallet-ethereum-transaction * upgrade pallet-loans * upgrade pool-system * upgrade pool-fees * upgrade pool-registry * upgrade liquidity-pools stuffs * avoid duplicated polkadot-sdk repos * minor fixes * upgraded runtime-common * CfgLocation to RestrictedTransferLocation * restricted tokens with NativeIndex for native fungibles * rename dependency * upgraded development-runtime * fix partially benchmarks * fix benchmarks * overpass xcmp-queue integrity tests * minor coments * upgrade altair & centrifuge * remove some benchmarking pallets that are not needed * fix runtime upgrades * upgrade integration-test proc * upgrade integration-tests framework * upgraded all tests except liquidity pools * 99% upgraded liquidity-pools tests * fix lookup * cargo fmt * taplo fmt * using nightly cargo in CI * restore set_balance as it was * allow nightly support in CI * use restricted_tokens again to fetch from investement portfolio * Install rust-src for docs * fix tests * remove unused restricted tokens * fix block rewards * fix WrongWitness for some tests in IT * fix liquidity-pools * minor fixes * fix clippy * remove unneeded tests * feat: Update client to Polkadot v1.7.2 (#1844) * wip: update client * chore: update crate versions * chore: update anchor rpc api * chore: remove rewards, pools rpc * chore: compile with development runtime * fix: client for all runtimes * fix: build spec * feat: update relay docker images * fix: apply deprecation of export genesis state to scripts * fmt: taplo * refactor: use xcm v4 sugar * fix: revert tmp change in local para run * refactor: simplify xcm v4 locations in chain spec * cargo fmt * fix clippy * feat: Polkadot v1.7.2 migrations (#1849) * feat: add hold reason migration * feat: centrifuge migrations * feat: altair migrations * feat: dev + demo migrations * fix: clippy * fix: build * fmt: fix using nightly * last William iteration review * increase passed blocks * use rococo instead of polkadot-test-runtime * fix tests * remove line * dirty fix to fix Hrmp test issue * use default weights for treasury * remove getrandom unused dep * upgrade to last polkadot-sdk version * feat: `CheckMetadataHash` extension * fix it (#1866) * fmt: taplo * refactor: signed ext order * fix: signed ext order for ITs * IT: more support for router tests (#1885) * move routers to its own module * remove outdated markers * for all runtimes * remove previous tests * minor fixes * v0.11.2 rc * panic if event is not found in the expected blocks (#1880) * fix: ci * remove generic module (#1887) * revert check metadata hash disable * fix: disable metadata hash check for integration tests --------- Co-authored-by: lemunozm <[email protected]> --------- Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Frederik Gartenmeister <[email protected]> Co-authored-by: lemunozm <[email protected]>
fix: adapt passthrough router fix: docs and hash chore: rm unused test Release v0.11.3 (#1886) * Switch to full wasm execution * feat: use correct encoded size. * fix: lint * refactor: reduce dev session duration from 6h to 2min * chore: update centrifuge weights * chore: update dev weights * chore: update altair weights * chore: update frame_system weights * fmt * fix: re-enable frame_system benches * chore: bump spec version * fmt: revert using latest nightly * v0.11.3-rc1: revert checkmetadata ext * feat: `CheckMetadataHash` extension (#1865) * set dependency versions * upgrade libs/* * upgrade pallet-anchors * upgrade pallet-fees * upgrade pallet-bridge * migrate simple pallets * migrate pallet-order-book * migrated collator-allowlist & swaps * upgrade rewards * upgraded pallet-mux * upgrade transfer-allowlist * fix hold reason in restricted tokens * simplify set_balance removing the holding part * upgrade restricted-xtokens * upgrade some pallets * upgrade pallet-ethereum-transaction * upgrade pallet-loans * upgrade pool-system * upgrade pool-fees * upgrade pool-registry * upgrade liquidity-pools stuffs * avoid duplicated polkadot-sdk repos * minor fixes * upgraded runtime-common * CfgLocation to RestrictedTransferLocation * restricted tokens with NativeIndex for native fungibles * rename dependency * upgraded development-runtime * fix partially benchmarks * fix benchmarks * overpass xcmp-queue integrity tests * minor coments * upgrade altair & centrifuge * remove some benchmarking pallets that are not needed * fix runtime upgrades * upgrade integration-test proc * upgrade integration-tests framework * upgraded all tests except liquidity pools * 99% upgraded liquidity-pools tests * fix lookup * cargo fmt * taplo fmt * using nightly cargo in CI * restore set_balance as it was * allow nightly support in CI * use restricted_tokens again to fetch from investement portfolio * Install rust-src for docs * fix tests * remove unused restricted tokens * fix block rewards * fix WrongWitness for some tests in IT * fix liquidity-pools * minor fixes * fix clippy * remove unneeded tests * feat: Update client to Polkadot v1.7.2 (#1844) * wip: update client * chore: update crate versions * chore: update anchor rpc api * chore: remove rewards, pools rpc * chore: compile with development runtime * fix: client for all runtimes * fix: build spec * feat: update relay docker images * fix: apply deprecation of export genesis state to scripts * fmt: taplo * refactor: use xcm v4 sugar * fix: revert tmp change in local para run * refactor: simplify xcm v4 locations in chain spec * cargo fmt * fix clippy * feat: Polkadot v1.7.2 migrations (#1849) * feat: add hold reason migration * feat: centrifuge migrations * feat: altair migrations * feat: dev + demo migrations * fix: clippy * fix: build * fmt: fix using nightly * last William iteration review * increase passed blocks * use rococo instead of polkadot-test-runtime * fix tests * remove line * dirty fix to fix Hrmp test issue * use default weights for treasury * remove getrandom unused dep * upgrade to last polkadot-sdk version * feat: `CheckMetadataHash` extension * fix it (#1866) * fmt: taplo * refactor: signed ext order * fix: signed ext order for ITs * IT: more support for router tests (#1885) * move routers to its own module * remove outdated markers * for all runtimes * remove previous tests * minor fixes * v0.11.2 rc * panic if event is not found in the expected blocks (#1880) * fix: ci * remove generic module (#1887) * revert check metadata hash disable * fix: disable metadata hash check for integration tests --------- Co-authored-by: lemunozm <[email protected]> --------- Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Frederik Gartenmeister <[email protected]> Co-authored-by: lemunozm <[email protected]> Refactor: Decoupling routers from gateway (#1891) * decoupling routers from gateway. Use Vec<u8> * gateway adapted * fix imports * move test msg to cfg-traits * fix imports feat: Open Gov for Development and Altair Runtimes (#1828) * chore: add OpenGov pallets to root toml * feat: add OpenGov tracks to runtime common * feat: add technical committee migration tc: fix clippy * refactor: move origins from primitives to common common: fix typo common: fix test after HalfOfCouncil refactor * feat: add OpenGov to Development fix: complete Development OpenGov development: finalize opengov development: add proxy calls development: overwrite missing origins dev: fix treasury spender dev: remove crowdloan pallets dev: taplo fmt * feat: add OpenGov to Altair fix: complete Altair OpenGov fix: add missing technical fellowship weights Altair altair: finalize opengov altair: rename track tests altair: gather imports altair: remove crowdloan pallets altair: taplo fmt * feat: add OpenGov to Centrifuge chain fix: centrifuge toml chore: minor centrifuge chain improvements refactor: Centrifuge OpenGov centrifuge: finalize opengov centrifuge: fix cargo tml cfg: fix clippy test * Revert "feat: add OpenGov to Centrifuge chain" This reverts commit 4d78c10. * fix: clippy * tests: fix CouncilCollective import path * fix: technical committee migrations * altair: configure tc migration (incomplete) * dev: configure tc migration * chore: cleanup altair dep diff * chore: remove unused getrandom crate from runtimes * chore: add missing Altair TC members * refactor: move to_ppm and to_percent to utils * Merge remote-tracking branch 'origin/main' into feat/open-gov-2 * fix: issues after rebasing * fmt * fix: imports * feat: v0.12.0 for altair RU Altair: release v0.12.0 (#1896) * fix: altair chain spec * chore: update gov2 weights * chore: update dev OpenGov weights * fix: weight declaration Fix RustDocs deployment & codecov patch checks (#1870) * Fix deploy docs * Make patch checks also informational (not fail) LP Message with custom data format for serialization/deserialization (#1889) * test compiling * fix Domain serialization * fix issues * some reorganizations * using bincode 2 with non-std support * use custom trait * tests passing for lp * remove ENCODED_MSG constant * fix runtimes compilation * cargo fmt & taplo fmt * cargo clippy * minor change * add custom serializer passing all tests * no_std support * minor renames * cargo fmt * cargo clippy * minor comment change * cargo fmt extra Succeed when patch codecov fails (#1900) Liquidity pools: Add UTs to for `update_token_price()` (#1890) * add token price UTs * update IT * fix tests compilation remove some unused code investment-related (#1902) Release v0.13.0 (#1898) * chore: update deps to enable metadata hash check * chore: update srtool + add on-chain-release-build opts * chore: bump spec versions + cleanup migrations * fmt: taplo * ci: fix srtool fmt * ci: apply proper CLI * attempt2 to enable on-chain-release-build * change srtool build for chevdor/strool-actions action * desperate attempts to get it working * override workdir * Try to set permissions wide open before running docker * add RUST_BACKTRACE * revert to our GHA manual process & add enhancements * fix package name * Output information about srtool * Revert "desperate attempts to get it working" This reverts commit a882fd9. * fix: some scripts * fix bad colon on echo command * Fix missing colon and remove docker publish release * upload wasm to release * fix issue with gchr tags * more semicolon issues * unique name for delete untagged * fix delete_untagged * move delete_untagged under workflows * move delete untagged to manual runs * review bash syntax for wasm build * recover cache and limit sanity check build time --------- Co-authored-by: Guillermo Perez <[email protected]> cargo update (#1904) add uts (#1905) IT: Support `#ignore = reason` for `test_runtimes` macro (#1908) * support #ignore = reason for test_runtimes macro * fix clippy ci: disable checks for drafts (#1913) LP: Unitary testing for add_currency (#1912) * port add-currency tests into uts * remove LiquidityPoolsWrappedToken struct * remove unused code * fix compilation issues * fix clippy * fix clippy feat: adatpt to latest version LP: Unitary testing for all pending extrinsics (#1916) * allow and disallow currency tests * schedule_upgrade cancel_upgrade and update_tranche_token_metadata Feat: Adapt proxy settings (#1922) * feat: adapt proxy settings * fix: match pattern Small CI improvements (#1924) LP: Unitary testing for inbound messages (#1927) * tests inbound messages * some cleanings * some adjustements fix: array size fix: clippy + fmt fix: update blake hash fix: compilation Refactor: Transform `TrancheCurrency` type in a tuple (#1926) * remove TrancheCurrency type * fix benchmarks * fix clippy fix: std import KeccakHasher
Description
CheckMetadataHashextension paritytech/polkadot-sdk#4274 to all runtimes which is necessary for the Polkadot Generic Ledger Appon-chain-relese-buildmodeTODO
on-chain-release-buildfeature--> Seems to be used by
srtoolper default which we use for release builds.Checklist: